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• ^ 1 INTRODUCTION 



ABSTRACT 

We present the rest-frame J- and H- band luminosity function (LF) of field galaxies, based 
on a deep multi-wavelength composite sample from the MUSYC, FIRES and FIREWORKS 
survey public catalogues, covering a total area of 450 arcmin^. The availability of flux mea- 
surements in the Spitzer IRAC 3.6, 4.5, 5.8, and 8 fim channels allows us to compute absolute 
magnitudes in the rest-frame J and H bands up to z 3.5 minimizing the dependence on 
the stellar evolution models. We compute the LF in the four redshift bins 1.5 < z < 2.0, 
2.0 < z < 2.5, 2.5 < z < 3.0 and 3.0 < z < 3.5. Combining our results with those already 
available at lower redshifts, we find that (1) the faint end slope is consistent with being con- 
stant up to z = 3.5, with a = — 1.05 ± 0.03 for the rest-frame J band and a = —1.15 ± 0.02 
for the rest-frame H band; (2) the normalization </)* decreases by a factor of 4-6 between 
z = and z ~ 1.75 and by a factor of 2-3 between z ~ 1.75 and z — 3.25; (3) the character- 
istic magnitude M* shows a brightening from z = to z ~ 2 followed by a slow dimming to 
z = 3.25. We finally compute the luminosity density (LD) in both rest-frame J and H bands. 
The analysis of our results together with those available in the literature shows that the LD is 
approximately constant up to z « 1, and it then decreases by a factor of 6 up to z = 3.5. 

Key words: galaxies: evolution - galaxies: statistics - galaxies: high-redshift - galaxies: 
luminosity function, mass function - infrared: galaxies 



In the current concordance model, galaxies are the result of con- 
tinuous mergers of dark matter halos driving baryonic matter as- 
sembly. In the last decades, simulations on halo occupation models 
have been able to quite accurately plot the formation of dark mat- 
ter clusters. However, big uncertainties still remain at the time of 
translating dark matter haloes to what can actually be detected with 
our telescopes. 

To this respect, the luminosity function (LF) of galaxies, i.e. 
the number density of galaxies per unit flux, is an extremely pow- 
erful tool to study the galaxy population and its evolution with cos- 
mic time. Specifically, the analysis of the LF at different rest-frame 
wavelengths can give us information on different aspects of our 
present view of the Universe. The UV-optical LF allows for the 
study of the content and the evolution of the star formation rates 
with cosmic time. On the other hand the near infra-red (NIR) LF, 
being less sensitive to the absorption by dust and dominated by the 
light of older stars, is a better estimator of the overall stellar mass 
assembly of galaxies and of its rate of growth with time, revealing 
itself as a good test-bench for halo models. 
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The local NIR LF is still not yet well determined. Although 
a number of measurements have been derived so far, there seems 
to be uncertainties especially for the faint end slo pe a . Estimates 
of th e slope a range f rom » -0 8 (Bell et alj |2003l : lEke et all 
l2005t) . to a « — L2 (llones et alj [2006). with a median value 
around -1 I Mobashe r. Sharpies. & Elli j " 19931 : [oiazebrook et al 



'1995"; 'Cowie etal."l996; Gardner et al. 1997 and Szokolv et ai 
1998; Kochanek et al. 2001; Cole et al. 2001 and Hill et al. 2010). 

At even larger redshift, the LF determinations (most of 
which are done in the rest-frame Ks band ) still suffer from sig- 
nificant uncertainties l lSaracco et Zl |2006|) . The fai nt-end slope 
seem s to be always compatible with a = — 1 jProry et al.l 
12003'; 'Pozzetti et aL"2003l: iDahlen et alj|2005l : ISaraccoet al .1120061: 
Cirasuolo et al. 2010) although these measurements suffer from the 
large uncertainties given by the limits in the depth of the photomet- 
ric catalogues available so far. There seems to be a general consen- 
sus however that the NIR LF doe s not significantly evolve to 2 ~ 1 
with respect to the local LF ( Cowie et al. 1996; Pozzetti et alj 
2003; Drory et al. 2003; Feulner et al. 2003; Dahlen et al. 2005). A 
brightening of the characteristic magnitude is instead found around 
z ~ 1.2 — 1.5 together with a decrease of the normalization, de- 
creas e that is seen up to 2 = 3 dSaracco et al.l200^ : ICirasuolo et al.l 
I2OIOI) . 
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In this paper we present the rest-frame J- and //-bands LFs 
and luminosity density (LD) of field galaxies, obtained from three 
deep photometric redshift surveys, namely MUSYC, FIRES and 
FIREWORKS, complemented by deep Spitzer 3.6, 4.5, 5.8, and 
8 /Ltm data. As discussed in e. g. ISerta et ah I ( l2007h . the combi- 
nation of the Planck spectral peak from low-mass stars, the mini- 
mum in the H~ opacity in stellar atmospheres and the molecular 
absorptions in the spectra of cold stars produce a maximum for 
the emission in the rest-frame NIR portion of galaxy spectra lo- 
cated at 1.6/im (the so called 1.6/im bump). Furthermore, the AGN 
light can contribute significantly to the rest-frame K band. Specif- 
ically, the contribution from the dust torus of the AGN can be in 
the rest-frame K band a factor of 10 larger than in the rest-frame 
J ba nd, and a factor of 4 larger than in the rest-frame H band 
(e.g.. |Polletta et al]|2008l) . The adoption of the rest-frame J and H 
bands makes thus the measurement of the LFs and LDs less sen- 
sitive to potential dust-obscured AGN contamination compared to 
measurements of the LFs in the rest-frame Ks, yet allowing us to 
sample a wavelength range dominated by stellar emission and very 
little affected by obscuration by dust. The combination of depth 
and wavelength coverage in the mid-IR out to 8/im allows us to di- 
rectly probe the rest-frame J and H bands out to 2 ~ 3.5, relying 
more on observational data rather than on stellar population mod- 
els, which are still significantly uncertain in the rest-fr ame NIR, 
due to different implementations of the TP-AGB phase jMarastonl 
l2005l : IConrov. Gunn. & Whitel2009l) . The total surveyed area sums 
to 450 arcmin^ with complete U-to-8/im coverage, reducing thus 
the effects of cosmic variance, which we estimate to give on aver- 
age a 15-20% contribution. 

This paper is organized as follows. In section 2 we present the 
data sets used for this work together with a description on how we 
recover photometric redshifts and we select galaxies from the full 
sample. Section 3 presents the three methods adopted to estimate 
the LF and its associated uncertainties. In section 4 we present our 
results, while our conclusions are summarized in section 5. 

Throughout this work, the adopted cosmology is SIa = 0.7, 
i^m = 0.3 and Ho = 70 Km/s/Mpc. All magnitudes are expressed 
in the AB system. 



2 THE SAMPLE 

For this work we used a total of seven public T^'a -selected cata- 
logues coming from three different deep multi-wavelength galaxy 
surveys covering the range from the optical to the Spitzer IRAC 
8 /im wave band: the MUlti -walelength Survey by Yale-Chile 
(MUSYC - iMarchesini eTZI [2009.). the Faint InfraRed Extra- 
galac tic Survey (FIRES - Labbe et alj2003l . lForster Schreiber et al.l 
l2006 l) and the GOODS Chandra Deep Field-South (FIREWORKS 
- IWuv ts et al. 2008). Although they have all been presented in 
IMarchesini et alj | |2009|) . for readers' sake these surveys will be 
briefly described in the following sections. 



2.1 MUSYC 

The deep NIR MUSYC survey consists of four 10' x 10' fields, 
namely, Hubble Deep Field-South 1 and 2 (HDFS-1, HDFS-2, 
hereafter), the SDSS-I030 field, and the CW-1255 field, observed 
with the Infrared Side Port Imager (ISPI) camera at the Cerro 
Tololo Inter-American Observatory (CTIO) Blanco 4 m telescope, 
for a total surveyed area of 430 arcmin^ . A complete description of 
the deep NIR MUSYC observations, reduction procedures, and the 



construction of the /^-selected c atalog with U-to-K photometry is 
presented in lOuadri et^ ( l2007h . Deep Spitzer-IRAC 3.6-8.0 fim 
imaging is also available for the four fields. The average total lim- 
iting magnitudes of the IRAC images are 24.5, 24.2, 22.4, and 22.3 
(3(7, AB magnitude) in the 3.6, 4.5, 5.8, and 8.0 /im bands, respec- 
tively. The A'-selected catalogs with IRAC photometry included is 
publicly available at http://www.astro.yale.edu/musyc The SDSS- 
1030, CW-1255, HDFS-I, and HDFS-2 catalogs are Ks band- 
limited multicolor source catalogs down to Ks,tot = 23.6, 23.4, 
23.7, and 23.2, for a total of 3273, 2445, 2996, and 2118 sources, 
over fields of 109, 105, 109, 106 arcmin^, respectively. All four 
fields were exposed in 14 different bands, U, B, V, R, I, z, J, H, K, 
and the four IRAC channels. The SDSS-1030, CW-1255, HDFS- 
I, and HDFS-2 K-selected catalogs have 90% completeness levels 
at ifs,9o=23.2, 22.8, 23.0, and 22.7, respectively. The final cata- 
logs used in the construction of the composite sample have 2825, 
2197, 2266, and 1749 objects brighter than the 90% completeness 
in the Ks band, over an effective area of 98.2, 91.0, 97.6, and 
85.9 arcmin'^, respectively, for a total of 9037 sources over 372.7 



2.2 FIRES 

FIRES consists of two fields, namely, the Hubble Deep Field- 
South proper (HDF-S) and the field around MS 1054-03, a fore- 
ground cluster at z = 0.83. A complete description of the FIRES 
observations, reduction procedures, and t he construct i on of pho- 
tometric catalogs is presented in detail in iLabbe et"al] ( l2003h and 
iForster Schreiber et alj j2006l) for HDF-S and MS 1054-03 (here- 
after HDFS and MS- 1054), respectively. Both Jfs-s elected cata- 
logs were later augmented with Spitzer-IRAC data jWuvts et al.l 
l2007l : lToft et"ai][2007 )). The HDFS catalog has 833 sources down 
to Ks,tot=26.0 over an area of 2.5 x 2.5 arcmin^. The MS- 1054 
catalog has 1858 sources down to Ks,tot=25.0 over an area of 
5.5 X 5.3 arcmin^. The HDFS field was exposed in the WFPC2 
U300, B450, V606, 18 14 passbands, the ISAAC Js, H, and Ks 
bands, and the four IRAC channels. The MS-I054 /^s-selected cat- 
alog comprises FORSI U, B, V, WFPC2 V606 and 1814, ISAAC 
Js, H, and Ks, and IRAC 3.6-8.0 fim photometry. The HDFS and 
MS-1054 catalogs have 90% completeness levels at A's.9o=25.5 
and 24.1, respectively. The final HDFS and MS-1054 catalogs used 
in the construction of the composite sample have 715 and 1547 ob- 
jects brighter than the 90% completeness in the Ks band, over an 
effective area of 4.5 and 21.0 arcmin^, respectively. 



2.3 FIREWORKS 

In this work, we adopted the A's-selected catalog (dubbed FIRE- 
WORKS) of the CDFS field c onstructed based o n the publicly 
available GOODS-CDFS data bv lWuvts et alj ( l2008l) . The photom- 
etry was performed in an identical way to that of the FIRES fields, 
and the included passbands are the ACS B435, V606, 1775, and 
z850 bands, the WFI U38, B, V, R, and I bands, the ISAAC J, 
H, and Ks bands, and the four IRAC channels. The /Tg -selected 
catalog comprises 6308 objects down to Ks,tot=24.6 over a total 
surveyed area of 138 arcmin^; the variation in exposure time and 
observing conditions between the different ISAAC pointings lead 
to an inhomogeneous depth over the whole GOODS-CDFS field 
(hereafter CDFS). The final CDFS catalog used in the construction 
of the composite sample comprises 3559 objects brighter than the 
90% completeness level {Ks,9q = 23.7), over an effective area of 
113 arcmin^ with coverage in all bands. 
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Figure 1. Photometric redshifts compared to spectroscopic redshifts, for the four different SED sets. From top to bottom: EAZY default template set; EAZY 
default template set with the addition of Lya, Ha, H/3, [Oil] and [OIII] emission lines; Coleman, Wu and Weedman template set and EAZY SED set with 
dusty galaxy template (see text for details). AtAz/{l+z) ^nd c^^/jx+z) the average and the standard deviation of Az/{1 + z), respectively; ui^mad is 
the normalized median absolute deviation. 



2.4 Photometric redshift and star/galaxy separation 

The downloaded catalogues all come with photometric redshift in- 
formation; spectroscopic redshifts are also available for a small 
fraction of galaxies (around 10% of the whole sample). However, 
we re-comp uted photometric redsh ifts, using the publicly available 
EAZY code jBrammer et al .|2008[) , and adopting four different sets 
of Spectral Energy Distribution (SED) templates. The first SED set 
is the EAZY default template set; it c onsists of 5 SED templates 
built on the base of PEGASE models jFioc & Rocca-Volmerangj 
I2OO6I) , reproducing the colors of g alaxies in the semi-analytic mod- 
els by iDe Lucia & Blaizou ( l2007h . plus a template representing a 
50 My galaxy with heavy dust obscuration. The second set is com- 
posed by a modified version of the standard EAZY templates, with 
the addition of Lya, Ha, H/3, [Oil] an d [OIII] emission lines. The 
third i s a set of six templates based on lColeman, Wu, & WeedmanI 
( Il98(]h colors, include d in the Bayesian Photometric Redshift code 
(BPZ - iBem'tejlioOol ). The last set is an extension of the standard 
EAZY template set with the inclusion of a IGyr galaxy template, 
with T = 1 00 Myr and Ay = 3 mag , similar to the reddest tem- 
plate used in lBlanton & Roweia ( l2007h . 

For all the four cases, the same default template error function 
and iC'-band prior was adopted. 



Figure [T] shows the Zspec vs. Zphot plot for the four SED tem- 
plate sets used. The average Az/{1 + z) are respectively -0.01, 
-0.01, -0.01, and -0.02 for the EAZY, EAZY+hnes, CWW and 
EAZY-l-dust template sets when considering the full sample, and 
-0.05, -0.04, -0.02, and -0.07 when computed on the redshift range 
1.5 < z < 4.0. The standard deviations (Jaz/{i+z) are 0.12, 0.12, 
0.12, and 0.13 respectively for the full sample and 0.17, 0.17, 0.17, 
and 0.19 in the redshift interval 1.5 < z < A. The normalized 
median absolute deviation, ctnmaeEI is 0.03, 0.03, 0.04, and 0.03 
over the whole redshift range, and 0.07, 0.07, 0.09, and 0.08 in the 
redshift interval 1.5 < z < 4. The fraction of catastrophic pho- 
tometric redshift {n{-^J > 5ct) is 0.042, 0.046, 0.045 and 0.048 
respectively. 

We finally chose to adopt the standard EAZY template set, 
which is the one presenting the smallest deviation between spec- 
troscopic and photometric redshifts in the targeted redshift range. 

The separation between stars and galaxy was done with the 

^ The normalized median absolute deviation o"nmad. defined as 1.48 X 
median p(Az — median{Az)) /(I + z)|j , is equal to the standard devi- 
ation for a Gaussian distribution, and it is less sensitive to outlie rs than the 
usual definition of the standard deviation (e.g., lllbert et al.ll2006l) . 
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Figure 2. Two colours diagram for all the sources in the composite cata- 
logues. Colours from Pickles 1 1998) stellai' atmosphere models are shown 
as red diamonds. The red dashed line represents Eq.[T] which we adopted 
to separate stars from galaxies 



colour-colour diagram (17 — J) vs. (J — Ks) (see figurejll l. The 
same colours were also computed from the IPicklesI ( 1 1998 *) stellar 
atmosphere models, in order to improve the boundaries between 
stars and galaxy, especially for the reddest stars which fall out of 
the main sequence (see figure|2]for full details). 

Galaxies were selected among the object satisfying the rela- 
tion: 



(1) 



(J - Ks) ^ 0.145 • (C/ - J) - 0.45 

As a cross-check, we run the EAZY code with the IPicklesI 

( Il998h model stellar atmosphere and checked that the objects iden- 
tified as galaxies via the two-colour diagram had a greater than 
the obtained on the same object with the EAZY galaxy template 
set. This criteria was satisfied by all objects previously selected as 
galaxies with only 6.5% of the objects selected as stars showing 
a discordant value for the x^, giving confidence in our method to 
separate stars from galaxies. 

A star-galaxy separation was also available in the original 
FIRES and FIREWORKS public catalogues. This selection was 
based on spectroscopy, SED-fitting with stellar te mplates and vi- 
sual inspection of the object morphology CRudnick et al.l |2003 . 
l2003h . We verified that those objects selected as stars in the FIRES 
catalogue were actually falling in the correct region of our two- 
color plot. 

The availability of Spitzer IRAC data for all our sample allows 
us to compute absolute magnitudes in the rest-frame J and H bands 
with little dependance on the SED templates; in fact, as an extreme 
case, the rest-frame H band at z — 3.5 is shifted to the range 
6.7— 7.9/im, well bracketed by the IRAC channels 3 and 4, centered 
at 5.8 and 8/im. 

Our final catalogue is composed by a total of 14295 galax- 
ies, with redshifts determinations lo z = 6.2 and median redshift 
Zmed — 1, distributed over an effective area of 450 arcmin^. In the 
redshift range of interest, 1.5 < z < 3.5, there is a total of 3496 
objects, of which ~ 6% have spectroscopic redshifts. 



STY maximum likelihood l lSandage. T ammann. & Y ahill 19791) and 
the S tep- Wise Maximum Likelihood lEfstathiou, Elli s. & Petersonl 
Il988l) . 

The need to analyze composite samples with different photo- 
metric depth was overcome by applying standard techniques avail- 
able for each chosen method. Below we summarize them. 



3.1 1/Vmax 

The 1/Vniax m ethod, first introduced b y lSchmidll ( Il968l) . was then 
generalized by ' Avni & Bahcaiil ( Il980h to allow the simultaneous 
analysis of composite samples. This method presents a number of 
advantages: it is simple to code, it directly estimates the normaliza- 
tion of the LF, and it does not make any assumption on the spatial 
distribution of galaxies. The draw-back is that it is sensitive to the 
presence of clustering in the sample, affecting the estimate of the 
faint end. 

In a given redshift interval [zl, z2], the galaxy number density 
per unit magnitude in the k-th absolute magnitude bin <E>fc (M) is 
computed as follows: 



MM) = 



E 



AM ^ Vmax,i 



(2) 



where AM is the width of the magnitude bin and Ngai is the 
number of galaxies in the redshift interv al of interest. In the coher- 
ent analysis proposed bv lAvni & Bahcall ( .1980,) . Vmax,i is given 
by: 



dV 

dz 



dz 



(3) 



with Us the number of samples constituting the full cata- 
logue, Qj the apparent area in steradians corresponding the the j-th 
sample, dV/dz the co-moving volume element per steradian and 
Zmax,i,j the maximum redshift at which the i-th galaxy could have 
been observed within the flux limit of the j-lh sample. Standard 
Poisson errors were associated to each $fc(M). 

3.2 Step-wise maximum likelihood 

This method, developed bv lEfstathiou. Ellis. & Petersonl jl988l) . is 
based on a maximum likelihood estimate and with a non-parametric 
form. This method relies on the fact that the number density of 
galaxies $(A/, x) can be factorialized into a function which de- 
pends on luminosity only, (j>{M), and a factor depending on the 
position /(x) so that <I>(M, x) = (^(M) ■ /(x). In particular, this 
means that the normalization factor of $ is lost during the maxi- 
mization of the likelihood, and should then be determined in some 
other way. The LF $(M) is expressed as the sum over n steps as: 

$(A'/) = 0,, M, - AM/2 <M <M, + AM/2, i = 1, .., n (4) 

By minimizing the natural logarithm of the likelihood expres- 
sion, a recursive formula is found, allowing to compute the (ji^. In 
order to take into account the non uniform magnitude limits of our 
data sets, we adopted the modification of the recursive expression 
as presented in iHill et al.. C201(I) : 



3 METHODOLOGY 

For the measurement of the LF, we adopted t hree among the most 
widely used methods, namely the l/Vmax iSchmidtlll968h . the 



^AM^ 



Efcif W{Mi - Mk) 



Eklf [h(M, ~ M/.O/Efif ch,AMH{M, - Mf,k)] 



where W is a window function: 
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W{x) = 
and 

H{x) = 



1 if - AA//2 <x ^ AM/2 
otherwise 



1/2-X/AA4 




if a; ^ -AM/2 

if - AM/2 < a; ^ AM/2 

otherwise 



(5) 



(6) 



Uncertainties were computed by taking the square root of the 
first n diagonal elements of the invers e of the information matrix 
(see lEfstathiou. Ellis. & PetersoiJll988h . 

The SWML method does not compute the normalization it- 
self. For it, we rescaled the LF shape by applying the conserva- 
tion of the number of galaxies and usin g the information from the 
1/Vmax method, as in iHill et al.l Jioiol) . Specifically, given an ab- 
solute magnitude range, let A'^ be the number of galaxies included 
in the given magnitude interval. The redshift edges allowing the 
selected sub-sample to be observed are then computed, together 
with the associated maximum co-moving volume Vmax ■ The num- 
ber density of galaxies obtained from the SWML estimate is then 
rescaled in order to match the N/V,nax value. The above method 
can thus be considered as a 1 /Vmax LF computed on a single wide 
absolute magnitude bin. 
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Figure 3. Cosmic variance estimate for our survey geometry following 
iMoster et alj ^2010) . as a function of redshift and mass (left vertical axis) 
and absolute magnitude (right vertical axis). 



range of values compatible with the la uncertainties in the a and 
M* parameters. 



3.3 STY maximum likelihood 

The third tech nique employed to compute the LF is the paramet- 
ric method of lSandage. Tammann. & Yahill jl979l) . The advantage 
is that the resulting LF is not binned, but instead it is a continuous 
function. On the other side, the shape of the LF is constrained by 
the model adopted. Similarly to the SWML, the STYML assumes 
that the LF can be separated into a term depending on the spatial 
distribution and a term which depends on the luminosity distribu- 
tion, thus losing the normalization. 

The expression for the probability pi of seeing galaxy i in the 
sample can be written as: 



p oc 



MM) 



(7) 



where the explicit dependence of the bright and faint absolute mag- 
nitude limits Mb and Mf to the apparent magnitude rus of the orig- 
inal catalogue allows to take into account the non uniform photo- 
metric depth across the seven catalogues constituting the final sam- 
ple. 

The 4'{M) function was chosen to follow the lSchechteiHl97^ 
distribution: 

<I>{M) = (O.41nlO)0*lO"-^(^-^*-'^^'(^+") exp [_io«-4(«' -a^)] (8) 

where <jf>* is the normalization, a is the faint-end slope, and AI* is 
the characteristic magnitude indicating the change from the power- 
law to the exponential regime. The normalization factor (p* was 
obtained by imposing the conservation of the number of galaxies in 
the total sample. 

Confidence levels for a and A/* corresponding to 68%, 95% 
and 99% were computed from the ellipsoid of parameters defined 
by: 



In £ = In £„ 



0.5x^(iV) 



(9) 



where £ is the maximum likelihood function and Lmax its value at 
maximum, while x| ( A'^) is the /3-point of the distribut ion with 
N degrees of freedom ( lEfstathiou. Ellis. & Petersonll 19881) . Uncer- 
tainties on the normalization factor 0* were computed from the 



3.4 Cosmic variance and photometric redshift uncertainties 

The data set used for our measurement of the LF is the combination 
of seven catalogues, each one related to a different region of the 
sky. This allows to keep in principle the effects of cosmic variance 
to low levels. 

In this work, we included a more refined estim ate of cosmic 
variance following the recipe bv lMoster etakl ilOld) . A halo distri- 
bution model is used to relate the stellar mass to the dark matter 
halo as a function of redshift; the galaxy bias is then estimated 
via dissipation-less N-body simulations. The cosmic variance is 
first computed on dark matter haloes, and then converted to galaxy 
cosmic variance by applying the galaxy bias. This estimate was 
cross-checked with t he dif ferent evaluation of cosmic variance by 
iDriver & RobothamI feOloV Their work is based on direct compu- 
tation of the cosmic variance using A/* galaxies from the SDSS 
catalogue. The expression found is then generalized to any red- 
shift bin amplitude and mean value and to any geometry of the 
survey. We find that the two estimates, in the case of Af* galax- 
ies, are consistent within 70% in the lowest redshift bin, but differ 
up to a factor of 2.5 in the higher redshift ranges. As discussed in 
[Driver & RobothamI ( I2OI0I) . this discrepancy can be explained as 
the change in M* stellar mass value with redshift. 

The computation of the luminosity as a function of mass (or, 
more frequently, the computation of mass from the luminosity), 
necessary to obtain the values for cosmic variance is generally a 
non trivial task, involving the generation of synthetic SEDs based 
on different initial mass functions, which are then fitted on a per- 
galaxy basis. For our purposes of cosmic variance estimate in the fi- 
nal LF, we performed the conversion between galaxy baryonic mass 
Af and luminosity Lj^^ and Lhab a-posteriori on the LF, under 
the work hypothesis that the mass-to-light ratio can be considered 
constant over all the involved luminosity range and equal to its aver- 
age value. We adopted the mean value {M/L) = LGtofrM^/ Lq 
from lcole et al. I dBoil) for both the J and H bands. 

Figure |3] shows as a contour plot the values of the cosmic 
variance for our data as a function of redshift and galaxy mass, as 
computed using the Moster et al. cookbook. The values of cosmic 
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Figure 4. Luminosity function for the rest-frame J band, in the four redshift bins. Asterisks represent the 1/Vmax measurement, diamonds are from the 
SWML. The Schechter function obtained via the maximum likelihood is shown as the solid black line. The coloured area in each plot indicates the 1 a (68% 
confidence level) from the parametric maximum likelihood. The inset shows the 1, 2 and 3 a contours (corresponding to 68%, 95% and 98% confidence 
region) for the joint a-M* parameters from the ML analysis. The dashed line marks our 1.5 < 2 < 2.0 LF. 



variance range from 0.07 to 0.19 in the lowest redshift bin, from 
0.09 to 0.24 in the 2.0 < 2 < 2.5 bin, from 0.10 to 0.32 in the 
2.5 < < 3.0 bin and from 0.12 to 0.42 in the 3.0 < z < 3.5 red- 
shift bin. The recovered uncertainties have been added in quadra- 
ture to the standard errors computed in the 1/Knax and SWML 
methods, while the cosmic variance corresponding to M* has been 
added in quadrature to the error on 0*. 

The effects of photometric redshift errors have been studied 
via Monte Carlo simulations. Five hundred realizations of the LF 
in each redshift bin were computed. The redshift of each source 
in the original catalogue was randomly modified according to the 
gaussian standard deviation recovered from figure [TJ the absolute 
magnitude of each object was then modified accordingly. The dis- 
tribution of parameters of the recovered LF did not show any sys- 
tematic effect and the spread of the parameters was compatible with 
the photometric redshift errors, consistent with the Monte Carlo 
simulations performed in lMarchesini et all i2007h . 



z range 



M* 



(f)* (10""' Mag-i Mpc-^*) 



1.5-2.0 
2.0-2.5 
2.5-3.0 
3.0-3.5 



-1.24 
-1.12 
-1.17 
-0.92 



0.03 
0.03 
+0.11 
0.13 
+0.18 
0.22 
+0.42 
0.48 



-23.72 
-23.60 
-23.42 
-23.28 



+0.09 
0.06 
+0.14 
0.17 
+0.22 
0.23 
+0.33 
0.39 



11.31 
7.45 
9.73 
4.36 



+0.28 
0.10 
+0.53 
0.38 
+5.37 
2.33 
+8.61 
1.93 



Table 1. Schechter parameters for the J LF from the maximum likelihood 
analysis with one a errors, including uncertainties from cosmic variance. 



z range 



M* 



0* (10-4 Mag-l Mpc-3) 



1.5-2.0 
2.0-2.5 
2.5-3.0 
3.0-3.5 



-1.30 
-1.23 
-1.11 
-1.30 



0.04 
0.03 
+0.12 
0.07 
+0.19 
0.18 
+0.40 
0.49 



-24.03 
-23.94 
-23.74 
-23 



+0.06 
0.05 
+0.13 
0.12 
+0.21 
0.23 
+0.36 
-0.42 



8.79 
4.35 
6.32 
2.03 



+0.50 
0.20 
+0.68 
0.45 
+4.52 
1.36 
+8.50 
-2.04 



Table 2. Schechter parameters for the H LF from the maximum likelihood 
analysis with one a errors, including uncertainties from cosmic variance. 



4 J- AND //-BAND LUMINOSITY FUNCTIONS 

Both rest-frame J- and H-haxiA LFs were estimated in the redshift 
intervals 1.5 < z < 2.0, 2.Q < z < 2.5, 2.5 < ^ < 3.0, and 3.0 < 



2: < 3.5 with the three methods described in Sec. [3] In Table [3] and 
Table|4]we present our measurements obtained with the SWML and 
method, while the derived Schechter parameters in each 
filter and redshift range are summarized in Table[T]and Table|2] 
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Figure 5. Luminosity function in the H rest frame filter. Plot conventions are the same as for FigureO 



Figure |4] shows the LF for the rest-frame J filter in the four 
redshift bins. The number of objects used to construct the LF in 
each redshift bin are respectively 996, 419, 298 and 103. The three 
methods return consistent measurements of the LFs. 

In Figure [5] we show our measurement of the LF obtained in 
the H filter, for the same four redshift bins as for the J-band LF. The 
number of objects used to compute the LF is 996, 419, 298 and 103 
respectively for the 1.5 < 2: < 2.0, 2.0 < 2 < 2.5, 2.5 < 2 < 3.0 
and 3.0 < z < 3.5 intervals. To date, this is the first measurement 
of the rest-frame H-hand LF in the interval z £ [1.5, 3.5]. 



4.1 Comparison with previous works 

The m easurements by IPozzetti et alj ( |2003|) and ISaracco et al.l 
are the only previously measured rest-frame J-band LFs 
computed in redshift ran ges c omparable with ours. 

In IPozzetti et al.l ( iSot) the LF is computed for the rest- 
frame J and rest-frame K bands, usin g both spectroscopi c and 
photometric data from the K20 survey ( lcimatti etai]|2002h . Out 
of the three redshift ranges used to measure the LF (2 G 
[0.2, 0.65], [0.75, 1.3], [1.3, 1.9]), the highest interval is the only 
one which we can compare to. Their rest-frame J-band LF is shown 
by the magenta triangles in the left panel of Fig. |6] Our data allow 
us to compute the LF down to Mj = —20, around two magnitudes 
fainter than their limit; in addition, the area covered by our sam- 




-26 -21 



Figure 6. Comparison of IPozzetti et alj i2003h (magenta triangles - left 
panel) andl Saracco et alj i200d) LFs (blue squares and blue solid line) with 
our measurements (black symbols: asterisks for 1/Vmax, diamonds for the 
SWML, dash-dotted line for the Schechter parameterization). The left panel 
refers to z 1.5 LF, while the right panel to the 2 3 LF. 



pie is approximately 10 times larger than the area covered by the 
K20 survey, allowing us to estimate the bright end of the LF more 
robustly and to a 0.5 mag brighter limit. In the overlapping abso- 
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range 



Mj\AB] SWMLm^mag-'-Mpc- 



1 / Vmax m„rnag-^ Mpc- 



Mh [AB] S W ml [hf □mag - 1 Mpc 



1/ Vmax [h^omag- '-Mpc- 



1.5<2<2.0 -25.51 
-25.01 
-24.51 
-24.01 
-23.51 
-23.01 
-22.51 
-22.01 
-21.51 
-21.01 
-20.51 



2.0 < 2 < 2.5 



2.5 < 2 < 3.0 



3.0 < 2 < 3.5 



-25.17 
-24.67 
-24.17 
-23.67 
-23.17 
-22.67 
-22.17 
-21.67 
-21.17 

-25.19 
-24.69 
-24.19 
-23.69 
-23.19 
-22.69 
-22.19 
-21.69 

-24.93 
-24.43 
-23.93 
-23.43 
-22.93 
-22.43 



4.78 (±3, 
2.87 (±0, 

1.08 (±0, 
2.90 (±0, 
4.44 (±0, 
5.90 (±0, 

9.09 (±0, 
1.26 (±0, 
1.61 (±0, 
2.08 (±0, 
2.29 (±0, 



30) ± 3.42 X 
77) ± 0.94 X 
14) ± 0.25 X 
25) ± 0.59 X 

35) ± 0.90 X 
49) ± 1.20 X 
84) ± 1.38 X 
14) ± 0.20 X 
20) ± 0.25 X 

36) ±0.41 X 
66) ± 0.69 X 



1.36{±0.52) ±0.61 X 10- 
4.52 (±0.91) ± 1.44 X 10" 
1.15 (±0.15) ±0.32 X 10- 
2.24 (±0.25) ±0.61 X 10" 
4.08 (±0.47) ± 1.11 X 10- 
4.22 (±0.64) ± 0.92 X 10- 
5.52 (±0.93) ± 1.27 x 10- 
6.51 (±1.28) ± 1.50 X 10- 
7.58 (±2.12) ± 2.31 X 10- 

8.41 (±3.73) ±4.62 x 10- 
2.82 (±0.76) ± 1.19 X 10- 
8.45 (±1.54) ±3.14 x 10- 
1.68 (±0.27) ± 0.60 X 10- 
3.78 (±0.53) ± 1.33 x 10- 
7.48 (±1.07) ± 1.88 X 10- 
1.03 (±0.17) ±0.27 X 10- 
1.14 (±0.29) ±0.38 X 10- 

2.64 (±1.25) ± 1.67 X 10- 
3.81 (±1.42) ± 2.13 X 10- 
9.80 (±1.67) ± 4.43 x 10- 
1.68 (±0.14) ±0.72 X 10- 
2.19 (±0.25) ±0.95 x 10- 
2.55 (±0.47) ±0.83 x 10- 



5.65 (±3.99) ± 4.13 x 10- 
3.39 (±0.98) ± 1.16 X 10- 
1.27 (±0.19) ±0.30 X 10- 
3.42 (±0.31) ±0.71 X 10- 
5.03 (±0.38) ± 1.01 X 10- 
5.82 (±0.43) ± 1.17 X 10- 
7.80 (±0.62) ± 1.13 X 10- 
9.39 (±0.84) ± 1.41 X 10- 



1.61 (±0.66) ±0.77 X 10- 
5.36 (±1.20) ± 1.78 X 10- 
1.38 (±0.19) ±0.39 X 10- 
2.65 (±0.29) ±0.71 x 10- 
4.48 (±0.48) ± 1.20 x 10- 
4.36 (±0.58) ±0.89 x 10- 
4.24 (±0.59) ±0.89 x 10- 
7.48 (±1.41) ± 1.67 X 10- 



1.61 (±0.66) ±0.84 X 10- 
4.43 (±1.11) ± 1.81 X 10- 
1.21 (±0.21) ±0.44 X 10- 
2.19 (±0.36) ±0.79 x 10- 
4.35 (±0.58) ± 1.52 x 10- 
5.80 (±0.73) ± 1.40 X 10- 



1.83 (±0.75) ± 1.07 X 10- 
2.51 (±1.13) ± 1.54 X 10- 
7.45 (±2.23) ±3.83 x 10- 
1.54 (±0.38) ± 0.75 x 10- 
2.82 (±1.35) ± 1.79 x 10- 



Table 3. Luminosity function values obtained with the SWML and 1/ Vmax 
methods in the four redshift bins for the J filter. The first error value refers 
to the error estimated via the information matrix for the SWML method and 
to the Poisson error for the l/Vmax method, while the second term is the 
cumulative error taking into account also cosmic variance uncertainties. 



1.5 < 2 < 2.0 -25.63 
-25.13 
-24.63 
-24.13 
-23.63 
-23.13 
-22.63 
-22.13 
-21.63 
-21.13 
-20.63 

2.0 < 2 < 2.5 -25.29 
-24.79 
-24.29 
-23.79 
-23.29 
-22.79 
-22.29 
-21.79 
-21.29 

2.5 < 2 < 3.0 -25.60 
-25.10 
-24.60 
-24.10 
-23.60 
-23.10 
-22.60 
-22.10 



3.0 < 2 < 3.5 



-25.25 
-24.75 
-24.25 
-23.75 
-23.25 
-22.75 



4.12 (±2.85 
3.71 (±0.79; 
1.24 (±0.14; 
2.74 (±0.23; 

3.92 (±0.32; 
5.04 (±0.44; 

8.61 (±0.8i; 
1.04 (±0.12; 
1.53 (±0.19; 

2.03 (±0.34; 

2.26 (±0.6i; 

2.04 (±0.55; 
4.06 (±0.80 
1.23 (±0.15; 
1.76 (±0.21; 

3.46 (±0.44; 

3.62 (±0.57; 

5.04 (±0.88; 
5.56 (±1.14; 

6.27 (±1.69; 

5.69 (±2.93; 
1.91 (±0.58; 
5.21 (±1.1 

1.05 (±0.18; 
2.20 (±0.35; 

3.93 (±0.65; 
5.56 (±1.03; 
7.23 (±1.51; 

1.40 (±1.02; 
3.32 (±1.13; 
6.10 (±1.98; 
4.69 (±2.10 

1.47 (±0.62; 
1.35 (±0.54; 



±2.95 X 10-' 
± 1.05 X 10- 
±0.27 X 10- 
±0.56 X 10- 
±0.80 X 10- 
± 1.04 X 10- 
± 1.32 X 10- 
±0.17 X 10- 
±0.27 X 10- 
±0.39 X 10- 
±0.64 X 10- 

± 0.74 X 10- 
± 1.28 X 10- 
± 0.34 X 10- 
± 0.48 X 10- 
±0.96 X 10- 
± 1.06 X 10- 
± 1.18 X 10- 
± 1.44 X 10- 
± 1.84 X 10- 

±3.46 X 10- 
±0.84 X 10- 
± 1.99 X 10- 
±0.39 X 10- 
±0.79 X 10- 
± 1.43 X 10- 
± 1.54 X 10- 
±2.12 X 10- 

± 1.17 X 10- 
± 1.79 X 10- 
±3.23 X 10- 
±2.87 X 10- 
±0.87 X 10- 
± 0.65 X 10- 



5.69 (±4.02) ±4.16 X 10 
5.08 (±1.20) ± 1.53 X 10 
1.69 (±0.22) ±0.38 x 10 
3.73 (±0.33) ±0.77 x 10 
4.91 (±0.37) ±0.99 x 10 
4.86 (±0.38) ±0.98 x 10 
6.19 (±0.50) ±0.90 X 10 
7.49 (±0.73) ± 1.16 X 10 



2.95 (±0.89) ± 1.15 > 
5.89 (±1.26) ± 1.92 > 

1.74 (±0.22) ±0.48 > 
2.02 (±0.23) ±0.55 > 
2.97 (±0.34) ±0.81 > 

3.75 (±0.51) ± 1.05 > 
4.36 (±0.71) ±0.99 > 
5.05 (±0.94) ± 1.23 > 



1.34 (±0.60) ±0.74 > 
3.49 (±0.97) ± 1.49 > 
6.98 (±1.37) ±2.64 > 
9.98 (±1.72) ±3.65 > 
2.67 (±0.39) ±0.95 > 
4.40 (±0.59) ± 1.54 > 
4.25 (±0.60) ± 1.06 X 10- 



1.10 (±0.55) ±0.72 X 10- 
1.93 (±0.73) ± 1.09 X 10- 

4.11 (±1.19) ±2.10 X 10- 
5.38 (±1.94) ±2.97 x 10- 
1.77 (±0.39) ±0.84 x 10- 
1.88 (±0.45) ±0.68 x 10- 



Table 4. Luminosity function values obtained with the SWML and 1 /Knax 
methods in the four redshift bins for the H filter The first error value refers 
to the error estimated via the information matrix for the SWML method and 
to the Poisson error for the l/Vmax method, while the second term is the 
cumulative error taking into account also cosmic variance uncertainties. 



lute magnitude range, the two measured LFs are in good agreement 
with in the 1 — a errors. 

ISaracco et alj ( l2003) estimated the J-band LF in three redshift 
ranges, namely z < 0.8, 0.8 < z < 1.9 and 1.9 < z < 4.0, using 
101, 100, and 84 galaxies, respectively, collected from the HDF-S 
data and complemented by V LT-ISAAC J,H and K imaging. The 
LFs from lSaracco et ai]( l2006h are compared to our measurements 
in Figure |6] At z « 1.5, the absolute magnitude ranges of the two 
determinations are quite different. Their lack of points at the bright 
end, presumably due to the very small field of the HDF-S compared 
to ours, is compensated by a deeper limit at the faint end. Their 
Schechter representation of the LF is flatter (a = —0.94) than our 
measurement of the LF at 1.5 < z < 2.0, and it presents a dim- 
mer characteristic magnitude (by '^l mag). However, whe n directly 
comparing the 1 /Vmax estimate from lSaracco et al. Iliooi) with our 
non parametric 1.5 < z < 2.0 LF, we find good agreement (see 
Figure (6] left panel) with measurements lying within the 1 a error 
bars. Despite these differences, our Schechter parameterization is 
substantially compatible also with their points. We would like to 
note that our composite catalogue provides an improved sampling 
of the bright end, resulting in overall better constrained Schechter 
parameters, due to the strong correlation among these parameters. 
In the highest redshift bin, the differences in the Schechter parame- 
ters are still present, with the faint-end slope a being the parameter 
showing the largest difference. As for the lower- redshift bin, when 
comparing our non parametric estimate with their points, we find a 
good agreement (see the right panel in Figure[6ll. 
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Figure 7. Evolution of the Schechter parameters of the J LF as a function 
of redshift. Estimates from this work are shown as filled blue stars. Mea- 
surem ents from the literature are also plotted ( vertical crosses: | Cole et all 
I2OOII : a sterisks: IPozzetti et all l2O03t crosses: iFeulner et alj |2003| : open 



squares: |Eke et all 2005t open circ les: IPahlen et al, l2005l : open diamonds: 
.Saracco eTalHoO^ ; open triangles: |jones et an200f ; filled circle: IhIu et alj 
2OIOI1 . Top panel shows the data for (p*. The dashed line represents Eg. 1101 
(see text for details); the middle panel presents the measurements for Al*, 
with the dashed line representing Eq. II II in the lower panel the faint end 
slopes a together with the average value (dashed line) are shown. The filled 
regions correspond to 95% confidence level intervals for the resulting fitting 
curves. 
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Figure 8. Evolution of the Schechter </>* , M* and a parameters from the H 
LF as a function of redshift. See caption to Figure |2]f or details. 



4.2 Discussion 

In Figure|7]we compare the Schechter parameters for the rest-frame 
J band obtained in this work with those available in the literature 
as a function of redshift. 

The upper panel shows the evolution of (fi* as a function of 
redshift, showing 0* monotonically decreasing with increasing z. 
By 2: ~ 2, (p* has decreased by approximately an order of magni- 
tude compared to the local values. The following parameterization 
was adopted to model the observed evolution of 0* with z: 



(10) 



where 6, 7, and /3 are the free parameters. The best-fit values 
obtained for the parameters of the rest-frame J band are 9j = 
2.6 ± 0.9 X 10-*^ mag-i Mpc'^, yj = 7.7 ± 0.3, /3j = 0.22. 
The P parameter was estimated together with the other two in the 
first instance of the best fitting procedure, and kept fix in a second 
iteration. The quoted errors refer to the second iteration. Equation 
llOl is plotted as a dashed line in the upper panel of Figure|7] 

In the middle panel of Fig.|7]we present the evolution of A/* as 
a function of redshift. The data show a brightening of A/* from the 
local universe to z ~ 2, foll owed by a slow dimming. In analogy to 
the LF shape bv lSchechtei] ( [T976) . it is then possible to introduce 
the following ad-hoc representation for M*{z): 

M*{z) = M [(1 + ^)/(l + 2*)]" exp [-(1 + z)/{l + z*)] (11) 

with fi, z* , and rj free parameters to be determined. By performing 
a least-square fit to the available data we obtain the following val- 
ues: fij = -37.6 ± 1.7 mag, z} = 17.5 ± 4.6, r]j = 0.16 ± 0.02. 
The resulting curve is plotted as a dashed line in the middle panel 
of Figure|7] 

The lower panel illustrates the behavior of a as a function of 
redshift. The error bars are here generally large and do not allow 
to properly evaluate the presence of evolution as a function of z. 
Therefore we limit ourselves to compute an average value, resulting 
ina = -1.05 ±0.03. 

Figure [8] shows the plots of the Schechter parameters as a 
function of redshift corresponding to the rest-frame H band. For 
this band, there are only two determinations of the LF from the 
literature, making it more challenging to determine the evolution 
with redshift. Despite this, we applied the same analysis done for 
the rest-frame J band, obtaining Or = 2.0 ± 1.3 x 10"'^ 



Figure 9. Luminosity density pj as computed from our Schechter param- 
eters (filled blue stars) and compared with the available data in the redshift 
range [0,3.0]. Grayed symbols indicate the luminosity density computed 
assuming an absolute magnitude limit of Mj = —20.0. The dashed line 
represents the LD obtained directly from Eq.[l2]in terms of Eg. llOl and ll II 
with 95% confidence level indicated by the filled region. Plotting symbols 
same as for Fig.|2] 



Figure 10. Luminosity density pj/ as computed from our Schechter param- 
eters (filled black stars) and compared with the available data. See caption 
to Fig.|9]for details. 



\MfJ-H ^ -40.0 ± 9.3, z*H = 13.9 ± 6.3, rjH = 0.19 ± 0.06 for 
Eg. II Hand a = — 1.15 ± 0.02. The resulting best-fits are shown as 
dashed curves in Figure [S] 



4.3 Luminosity densities 

Here we present our measurements of the luminosity density (LD). 
Given the coupling between the Schechter parameters a and A/*, 
the luminosity density is a robust tool to characterize the evolution 
of the LF with cosmic time. The LD was obtained in the standard 
way, i.e. as: 



pJ = 



L^{L)dL = r(2 + Q)L>* 



(12) 



Mpc 



7h 



mag 

7.8 ± 0.5, I3h = 0.30 for the parameters of Eq. 



where the last equality holds when assuming a Schechter 
parametrization for the ^{L). This means that we are assuming 
that the Schechter distribution is a good representation of the un- 
derlying luminosity function. Figure |9] shows the evolution of the 
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Filter z range 



logp 



log p* 



logp 



J 1.5-2.0 
2.0-2.5 
2.5-3.0 



20.26 ±0.08 
19.98 ± 0.12 
20.05 ± 0.18 



20.22 ± 0.09 
19.96 ± 0.13 
20.01 ± 0.20 



20.22 ± 0.09 
19.93 ± 0.13 
19.93 ± 0.24 



3.0-3.5 19.57 ±0.30 19.55 ± 0.32 19.45 ± 0.39 



An exercise similar to what done for the J-band luminosity den- 
sity, introducing our parameterizations, is shown as a dashed line 
in Figure [To] The agreement is quite good over the whole redshift 
range, although more measurements are necessary at intermediate 
redshifts (2 < 1.5). 



H 



1.5-2.0 
2.0-2.5 
2.5-3.0 
3.0-3.5 



20.30 ±0.08 
19.93 ± 0.12 
19.96 ± 0.21 
19.61 ± 0.32 



20.27 ±0.09 
19.90 ±0.14 
19.94 ±0.24 
19.57 ±0.38 



20.27 ±0.09 
19.87 ±0.14 
19.85 ± 0.28 
19.39 ± 0.52 



Table 5. Luminosity density in logarithmic scale and expressed in units 
of log[W Hz~^ Mpc~^], in the rest-frame J and H bands. Quoted er- 
rors include the effects of cosmic variance, which is the dominant source 
of random uncertainties. In the third column we report the luminosity den- 
sity computed using Eg. I12l and coiresponding to the black stars in Fig- 
ures |9] and 1101 the values of the luminosity density in the fourth col- 
umn (log p*) reflect the upper limit in absolute magnitude correspond- 
ing to Mii^ = —20, which we imposed in order to limit the effect of 
the uncertainties in the determination of a (grey stars in the same fig- 
ures). The values in the last column (log p) list the luminosity densities 
computed using the corresponding absolute magnitude limits at each red- 
shift bin and for each band, i.e., Mj = —20, —21, —21.5, —22, and 
J^H.lim = -20, -21, -22, -22.5 for the redshift intervals centered at 
2 = 1.75, 2.25, 2.75, 3.25, respectively. 



luminosity density in the J filter, while Figure[To]displays the cor- 
responding plot for the rest-frame H band. Values of the luminosity 
density at each redshift and for each filter are presented in Table[5] 

In order to be less sensitive to the derived faint end slope of 
the LP, we also computed the luminosity density assuming two dif- 
ferent limiting absolute magnitudes. First, the luminosity density 
p was derived using the absolute magnitude limits of our survey 
in each redshift bin, i.e., Mj^um — —20, —21, —21.5, —22, and 
MH,iim = -20, -21, -22, -22.5, for the redshift intervals cen- 
tered at 2 = 1.75, 2.25, 2.75, 3.25, respectively. Second, the lumi- 
nosity density p* was derived assuming a limiting absolute mag- 
nitude equal to the brightest limit over the entire targeted redshift 
range, i.e. Mum ~ —20.0. The values of p* are also plotted in 
Figures|9]and[T0]as grey symbols. 

The overall plot of the J-band luminosity density shows a con- 
stant value for 2 < 0.8 — 1.0. At z ^ 0.8 — 1.0, the luminosity 
density starts to decrease down to 2 ~ 3.5, where pj is 16% of 
the 2 = value. This can be better visualized by comparing this 
plot with the top and middle panels of Figure [7] Here we see in 
fact that for 2 < 1 the decrease in number of galaxies is balanced 
by a brightening of the characteristic magnitude. At 2 > 1, both 
quantities decrease, with the resulting decrease of the luminosity 
density. 

Using the expression of Eg.llOland Eq.[TT]in Eq.[T2] it is pos- 
sible to obtain a functional representation of the luminosity den- 
sity. The dashed line in Figure |9] represents the luminosity density 
for the rest-frame J band obtained with this method and adopt- 
ing the best-fit values of the parameters previously recovered. The 
agreement with the points is good over the entire redshift range. We 
would like to stress that no best fit has been done using the data of 
the luminosity density. 

Similarly to the case of the LF, the luminosity density in the H 
filter has been poorly studied, so that it is more difficult to derive its 
evolution. Our data however indicate a decline with redshift of the 
LD, similar in shape to the one found in the J band, with a faster 
evolution from 2 — 3.5 to 2 — 1.5, followed by a much slower 
evolution, decreasing by a factor of 7 from 2 = to 2 = 3.5. 



4.4 Comparison with rest-frame if-band LF and LD 

In Fig. [TT] we present the evolution of the Schechter parameters 
0* (top panel) and M* (bottom panel) for the rest-frame K band, 
collected from the literature (coloured points - see legend for 
details) and overplotted to the corresponding parameterization 
of the rest-frame J band (taken from Fig. [7] black line). The 
previously measured rest-fra me J^ -b and LFs are tak e n from 
Mobasher, Sharpies, & Elli j (Il993l): lOlazebrook et al.l ( |l995 | ) : 
Cowieetal. (1996); Gardner et al. ('1997^; 'Szokoly et al.' ('l998); 
Lovedav (2000); Kochanek et aL (2 Q0 1); ,Cole et aL (.2001 ); 
Isolzonella, Pello, & Maccagni Booj ): FeulneretalJ hoOJS): 



'Huang et al . (i2003i); Pozzetti et al.l l!2003i); iKashika wa etd 
(2003); Sa racco et al.l (2006); Caputi et al. (2006); Cirasu olo et al 



( 120071120 1 3 ) - see also Table 5 in lSaracco et alj ilOOdt) which we 
used as reference for the literature works. 

Using the rest-frame Ti'-band data, we performed the same 
analysis as done for the rest-frame J and H bands, modeling the 
evolution of the Schechter parameters with redshift using Eq. [TO] 
and[TT] The best-fit values of the parameters obtained in modeling 
of the JC-band points are: Ok = 3.8 ± 0.1 x 10"^ mag~^ Mpc"^, 
jK = -0.11 ± 0.02, Pk = -2.2 ± 0.1 for the parameters of Eq. 
\M MA- = -29.6 ± 0.4, z'k = 113 ± 46, riK = 0.058 ± 0.007 
for Eq.[TT) and qk ~ —1.12 ± 0.16. The resulting curves are plot- 
ted in Fig.[TT]as red dot-dashed curves, together with those already 
discussed for the rest-frame J band (black dashed curves). 

The rest-frame Tf-band characteristic density, <l)l^, decreases 
by a factor of ~ 15 from 2 ~ to 2 ~ 3.3, about twice as much 
than the redshift evolution of the rest-frame J-band characteristic 
density, (j}j . Specifically, the data in the rest-frame J band indicate 
an evolution with redshift broadly consistent to the rest-frame K 
band out to 2 ~ 2.3, and a milder evolution at 2 > 2. Quanti- 
tatively, while <l)j decreases by a factor of ~ 2 from 2 ~ 1.5 to 
2 ~ 3.3, decreases by a factor of ~ 5 over the same redshift 
interval, although these differences are significant only at the 2 a 
level. 

Differences between the rest-frame J band and the rest-frame 
K band are also found when comparing the evolution with red- 
shift of the characteristic magnitudes and AfJ (bottom panel 
of Fig. II It. At 2 < 1, similar evolutions with redshift of AI^ and 
AIj are found, with the characteristic magnitudes brightening by 
~ 0.8 mag from 2 ~ to 2 ~ 1. However, for 2 > 1, 
is monotonically decreasing, brightening by 0.5 mag in the range 
2 € [1.5, 3.3], whereas M} shows a small dimming (if any) over 
the same redshift interval, after reaching its brightest value some- 
where in the redshift interval 1.5 < 2 < 2.5. We note that, also 
in the case of A/*, these differences are only marginally significant 
(within 2 a). 

Figure [12] shows the comparison of the evolutions with red- 
shift of the rest-frame K- and J-band LDs. As the differences in the 
evolutions with redshift of 0* and Al* between the J and K bands 
go in opposite directions, in the computation of the LDs these dif- 
ferences partly cancel out. As shown in Figure [T2] the evolution of 
the rest-frame AT-band LD with redshift is qualitatively similar to 
the evolution with redshift of the rest-frame J-band LD, decreasing 
by a factor of almost ~ 2 from 2 ~ to 2 ~ 2.5. 



The rest-frame J- and H-band luminosity functions to z = 3.5 11 




A Mobasherl 993 
O Glazerbrookl 995 
V Cowiel 996 
[> Gardnerl 997 
Szokolyl 998 
S Loveday2000 
♦ Kochanek2001 

■ Cole2001 

■ Bolzonella2002 
O Feulner2003 

▲ Huang2003 
T Pozzetti2003 
~ Kashika wa2003 
O Saracco2006 
Z Caputi2006 
□ Cirasuolo2007 
► Cirasuolo20 1 



2 
z 



Figure 11. Top panel: comparison between th e evolution w i th red shift of the Schechter param eter 0* in the rest-frame J and K (coloured points - see the 
legend on the right for details) bands. For the ICaputi et al.l j2006h and ICirasuolo et al] j2010l) LFs, the parameter values recovered from a Schechter fit to 
the l/Vmax LFs are identified by an additional circle and are displaced by an arbitrary amount of 0.05 in redshift to increase readability. The black dashed 
curve shows the best-fit pa rameterizations obta ined for the rest-frame J b and. The red and the blue curves mark the best-fit parameterization for the K band 
using the original ML froml Caputi et alj j2006l) and lCrrasuolo et al] feOlOl) and that using the points from the Schechter fit to their 1/ Vmax LFs, respectively. 
The hatched regions delimit the 68% confidence level for the fit parameters for the rest-frame J band (grey), K band (orange), and K band with Schechter 
parameters obtained from fitting the l/Vmax LFs (cyan), respectively. 95% confidence intervals are also plotted as coloured solid black, red, and blue curves, 
respectively. Bottom panel: evolution with redshift of the rest-frame J- and ii'-band characteristic magnitude A/*. Same plotting conventions as top panel 
apply. 



The evolution of the A'-band LF (and, c onsequently, LP) a t 
z > 1 is dominated by th e measurements bv lCaputi et alJ ( l2006h 
and lCirasuolo et al] ( 1201 Ol) . obtained adopting ad-hoc parameteri- 
zations for and M ^,with0^(z) oc exp( — z'''^) and M^{z) oc 
gVM (-ggg gq 2 and 3 in lCirasuolo et alj2010l for details). In order to 
verify that the derived redshift evolution in the rest-frame K band 
does not depend on the adopted parameterizations, we computed 
the Schechter function parameters by fitting their 1 /l/max estimates 
with a Schechter function with Mk, (^'k and qk as free parame- 
ters. The results are indicated in Fig.[TT]and Fig.ll2lbv an additional 
circle around the corresponding symbol, while our evolutionary- 
model best fits (Eg.llOlandlllt are shown by the blue curves. The 
values of the Schechter parameters recovered from the fit to the 
1/l^ax measurements are in broad agreement with those obtained 
from the ML analysis in combination with the adopted <j)*K{z) and 
M'^{z) parameterizations, although with larger error bars. Conse- 
quently, the evolution with redshift of (fix and inferred from 



the Schechter parameters obtained from the fit to the 1/Vmax mea- 
sureme nts partly overlap to that obtained using the or iginal ML 
data of lCaputietai] (2006*) and ICirasuolo etaU ilOld) . although 
with significantly larger errors. 

The final picture is that globally the evolutions of the rest- 
frame J— and A'— band LFs are (marginally) different within a 
confidence level not exceeding the 95% (i.e. roughly 2 a). The 
LDs in the two rest-frame bands are consistent up to z ~ 2.5. At 
z > 2.5, there seems evidence of a faster decrease with redshift 
of the rest-frame A'-band LD compared to the rest-frame J-band 
LD at 68% confidence level. However, the evolutions of the LDs 
in the two bands are comparable when considering the 95% confi- 
dence level. We note that the measurements of the A'-band LDs de- 
rived from the Sche chte r function fits to the 1/ Knax analysis from 
ICaputi et al] | |2006|) and lCirasuolo et al] ilOKj) are consistent with 
our estimated rest-frame J-band LDs within the errors. More ac- 
curate measurements of the LFs at z > 2.5 in both the J and K 
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Figure 12. Comparison between the evolution witli redshift of the LD in the rest-frame J (black curves and gray hatched regions - same plotting conventions 
as for Fig. [tJ and K bands (coloured points - same plotting conventions as for Fig. lilt . The hatched regions delimit the 68% confidence levels for the fit 
parameters, while the external coloured solid curves delimit the 95% confidence levels. 



bands are required to further investigate possible differences in the 
redshift evolution of the LDs in the two NIR bands. 

Figures [TT] and [12] show the presence of significant scatter in 
the measurements of the rest-frame K-hwA characteristic mag- 
nitudes A/k and LDs from different works (up to a facto r of 
-^2-3 when compar ing the works of K ashikawa et al.l [20031 and 
ICirasuolo et"ai]|20roO . Whereas it is difficult to assess the sources 
of such scatter and beyond the scopes of this work, we note that 
potential sources for such a significant scatter are twofold. 

First, any direct measurements of the rest-frame A'-band LF 
and LD at z > 0.6 require observations at wa velengths longer tha n 
2.2 ^J,m, provid e d by, e.g., S nitzer-IRAC. As ICaputi et air i [200^ . 
ICirasuolo et"al] ( |2007|) . and Icirasuolo et al.l (|20lrf) are the only 
previous works targeting the rest-frame K band including IRAC 
data in their analysis, all the other works do not directly probe the 
rest-frame K band, but instead rely on stellar population synthe- 
sis models or empirical templates to extrapolate the rest-frame K- 
band magnitudes of galaxies. Therefore, some of the scatter ob- 
served in the measurements of the rest-frame A'-band LF parame- 
ters and LDs at 2 < 2.5 could be due to differences in the adopted 
stellar population synthesis models, suc h as differences in the im- 
plementation of the TP-AGB phase (i.e.. lBruzual & Charlot|[2003l : 
lMarastorill2005l) . Note that observations at wavelengths longer than 



provided by IRAC would be needed to directly probe the rest-frame 
A band at z > 2.6, so all measurements of the rest-frame A-band 
LF and LD at z > 2.6 rely on stellar models. 

Second, emission associated to obscured (e.g., type-II) AGN 
(e.g., continuum emission from the dusty torus) can be signifi- 
cant in the rest-frame A band and at longer wavelengths. At low 
redshifts (z < 0.15), nuclear contamination in the A band from 
(obscured) AGN has been shown to be ~25% on average in ra- 
dio galaxies, and as high as 40 % (on average) in broad-lined ra- 
dio galaxies dlnskip et al]|2010l) . AGN contamination are gener- 
ally much smaller in the rest-frame J or H bands, w ith an av- 
erage contamination of ~3-4% (e.g., iFlovd et alJboOSi) . We note 
that, while these levels of AGN contamination have been derived 
for radio-loud AGNs, which represent only ~10% of the overall 
AGN population, radio-quiet AGNs are expected to be character- 
ized by similar levels of contaminati on, as a result of the similar 
orien tation-based unification scheme ( lAntonuccill993l ; rH6nig et al] 
I2OO6I) . Moreover, evidence for an increasing fraction of AGN as a 
function of stellar mass at z > 2 has been recently found, with an 
AGN fraction as h igh as 70% for massive galaxies at 2.0 < z < 2.7 
jKriek et al.|[2007i) . and potentiall y even higher AGN fra ctions in 
massive galaxies at 3 < z < 4 iMarchesini et aljboid) . As the 
rest-frame A band is directly probed by IRAC 3.6, 4.5, 5.8, and 



The rest-frame J- and H-band luminosity functions to z = 3.5 13 



8 /im channels at z ~ 0.6, 1, 1.6, and 2.6, respectively, the AGN- 
associated emission can potentially contaminate the measurements 
of the rest-frame A'-band LFs and LDs, especially at the bright end, 
potentially contributing to the observed scatter in the rest-frame K- 
band measurements of M* and LD. An accurate quantification of 
the contamination from AGN to the rest-frame K band would re- 
quire a detailed modeling of the SEDs from the X-ray to the in- 
frared associated to each object (see, e.g., lLusso et al which 
is beyond the scope of this work. 



5 CONCLUSIONS 

In the present work, we used a composite sample constructed from 
deep multi-wavelength publicly available photometric catalogues 
from the MUSYC, FIRES and FIREWORKS survey. The avail- 
ability of Spitzer-[RAC data in the 3.6, 4.5, 5.8, and 8/im channels 
allows us to robustly estimate the LFs and LDs in the rest-frame 
J and H bands with a minimum dependence on the SED tem- 
plates up to 2: = 3.5. Ours is the first measurement of the rest- 
frame Jf-band LF at 2 > to-date. We determined the LF with 
three independent methods, namely the 1/Vmax, the SWML ,and 
the STYML methods, finding that they agree well with each other. 
Uncertainties introduced by the c osmic varianc e were estimated us- 
ing two_distinctjiietiiods^_OM {2OIO) and the other 
bv lDriver & Robothaml ( [2oToh . We find that, for our data, the two 
approaches broadly agree. 

Our rest-frame J-b a nd LF is consistent with previous determi- 
nation bv ISaracco et 1a\ ( l2006l) . although the recovered Schechter 
parameters M* and a are consistent only at the 2cr level. This 
might be due to t he limited ranee in re st-frame magnitudes probed 
by the sample in lSaracco et al.l fcoOfih and by large errors due to 
cosmic variance given their small surveye d area. Our J - band LF 
is consistent also with the LF measured bv lPozzetti et al.l ( |2003|) at 
z ~ 1.5. 

We analyzed the evolution with redshift of the Schechter func- 
tion parameters, making full use of the data available from the lit- 
erature. We found that the faint end slope a of the LF is nearly 
constant over the whole redshift range, with aj = —1.05 ± 0.03 
and an = —1.15 ± 0.02 in the J and H bands, respectively. The 
characteristic density <j)* decreases by a factor of ~ 6 from 2: ~ 
to 2 = 1.75, and by a factor of « 3 from 2 ~ 1.75 to 2 = 3.25. 
We introduced a parameterization based on an exponential form 
for the evolution of (j)* as a function of 2. The fit of this function 
to the available data shows good agreement, especially for the rest- 
frame J band, where more data from the literature are available in 
the redshift range z € [0, 1], complementing our measurements at 
2 > 1.5. 

The characteristic magnitude M* is found to brighten from 
2 = 0tO2 ~ 2by~ 0.8 mag, whereas M* gets f ainter with 
increasing redshift at 2 > 2. We adopted a lSchechteil jl976l) -like 
expression for its description, resulting in a good representation of 
the observed evolution. 

We computed the LD in the rest-frame J and H bands, using 
the Schechter parameters previously determined. The LD is nearly 
constant up to 2 « 1 and decreases as a power-law by a factor of 
~ 6 from 2 « 1 to 2 = 3.25. 

We compared the evolution with redshift of the LFs and LDs 
in the rest-frame J and K bands. The Schechter parameters (j)* and 
M* in the rest-frame K band show different evolutions with red- 
shift with respect to the J band, although these differences are only 
marginally significant at the 2-a level. Specifically, the decrease of 



the characteristic density with redshift appears faster in the K than 
in the J band at 2 > 1.5. The characteristic magnitude in the rest- 
frame K band brightens with redshift over the whole redshift inter- 
val < 2 < 4, whereas A/j gets brighter from 2 = to 2 ~ 2, and 
then slowly gets fainter out to 2 = 3.5 (although a constant value 
of Mj al z > 1.5 is consistent within the uncertainties). Most of 
these differences cancel out when computing the LDs, with similar 
evolutions with redshift of the rest-frame J- and i^-band LDs out 
to 2 ~ 2.5. Evidence for a faster decrease with increasing redshift 
of the rest-frame A'-band LD at 2 > 2.5 seems present. However, 
these differences are significant only to a 95% level (2 a). Large 
errors at 2 > 2 in both the J and K bands prevent to firmly as- 
sess differences between the evolution with redshift of the J- and 
A'-band LFs and LDs. 

In order to further constrain the rest-frame J- and //-band 
LFs, a larger area is needed to better probe the bright end and re- 
duce the impact of field-to-field variations and low number statis- 
tics. Better photometric redshift estimates are also needed to im- 
prove the LF measurements at the bright end, by, e.g., reduc- 
ing the impact of catastrophic outliers. Th e recently publicly re- 
leased NEWFIRM Medium-Band Survey jWhitaker et"al]|201lh 
will provide the dataset to significantly improve on all of these as- 
pects. Ongoing very deep ground-based surveys, such as the Ultra- 
VISTA survejQ and space-based surveys with WFC3 on the Hub- 
ble Space Telescope, su ch as the CANDELS dOrogin et al.ll201 J : 
iKoekemoer et al.l I2OI ih and 3D-HST (Brammer et al. 2011, in 
prep.) surveys, will allow for much improved constraints of the 
faint end of the rest-frame NIR LF and of the contribution of low- 
luminosity galaxies to the total NIR LD. 
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